Fine-Tuning BERT-Based Pre-Trained Models for Arabic Dependency Parsing

With the advent of pre-trained language models, many natural processing tasks in various languages have achieved great success. Although some research has been conducted on fine-tuning BERT-based models for syntactic parsing, and several Arabic developed, no attention paid to dependency parsing. In this study, we attempt fill gap compare nine strategies, encoding methods We evaluated three treebanks highlight best options capture dependencies data. Our exploratory results show that AraBERTv2 model provides scores all confirm higher layers is required. However, adding additional neural network those drops accuracy. Additionally, found differences techniques give highest scores. The analysis errors obtained by test examples highlights four issues an important effect results: parse tree post-processing, contextualized embeddings, erroneous tokenization, annotation. This study reveals a direction future achieve enhanced